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ABSTRACT 

Hjw well a combination of j uni or /s eni or undergraduate 
grade point average (GPA) , scores on the Miller Analogies Test (MAT), 
and writing sample scores can predict success for students seeking a 
Master's degree in education was studied. Success was determined for 
the study by the student's graduate GPA and whether or not the 
student graduated from the program. Data were ootained for 142 
students. Their writing samples were scored independently by three 
experienced readers. Data indicated that the writing sample, in 
conjunction with Junior/Senior GPA, could be used with moderate 
success to predict undergraduate GPA for students in a Master's 
program in a school of education, although none of the variables 
could accurately predict whether or not a student would complete the 
program within 4 years. These results support previous findings that 
the writing score was a better predictor of graduate GPA than either 
the MAT score or undergraduate GPA. The study provides additional 
evidence that the use of authentic methods, such as a written 
statement, may be valuable in assessing a student's potential for 
program success. (SLD) 
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Using a Writing Sample to Predict Success 
in Masters Programs in Education 

by Jan Perney 
Nat ional -Louis University 

Obi ective 

The objective of this study was to determine how well a 
combination of junior/senior undergraduate grade point average, 
Miller Analogies Test (MAT) scores, and writing sam.ple scores can 
predict success for students who are seeking a masters degree in 
Education. For this study, success was determined by (1) the 
student's graduate grade-point average (GPA) and (2) whether or not 
the student graduated from the program. 

Perspectives 

Research studies have indicated that standardized test scores 
are of limited use to an admissions office that wants to predict 
performance of applicants to graduate Schools of Education. For 
the past thirty years the usefulness of standardized tests, such as 
the MAT, as predictors of ability in graduate Schools of Education 
has been limited (Ainsworth and Fox, 19S6; Gill and Marascuilo, 
1967) . Payne and Tuttle (1966) found predictive validity 
coefficients for the Miller Analogies Test that ranged from .00 to 
.59. A more recent study (de-Felix and Houston, 1986) indicates 
that there is a weak association between standardized entrance 
tests and indicators of success, particularly for minoritie.s. 
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' Today, with the movement toward assessment devices which are more 
authentic in nature, it seems reasonable to explore the possibility 
of emphasizing a writing sample, as opposed to a standardized test, 
as an admission criterion. 

Currently the MAT score and a student's junior/senior 
undergraduate GPA are used as the tv;o major criteria for admission 
to graduate study in the College of Education at a metropolitan 
university in the Midwest. It was hypothesized that the quality of 
students' writing might be more predictive of their performance in 
graduate school than the MAT, because most of the masters programs 
at the university emphasize the production of original v;ritten 
work. Much of the assessment for courses is based on essay exams 
and written papers, rather than students' responses to objectively 
scored test items. 

This current study is an attempt to replicate a study 
conducted at the same College of Education in 1993 with Masters of 
Arts in Teaching students and another in 1994 with on-campus 
Masters of Education students which found that the scores from a 
writing sample predicted graduate GPA more accurately than scores 
from the MAT. This study differs from the previous studies in that 
a sample of students from all masters programs in the College of 
Education was selected. 



MeLtlods 

The Office of Institutional Research at the university 
supplied a list of 6738 students who were seeking masters degrees 
and taking courses in the College of Education from September 1991 
to August 1993. The list of students was then stratified with 
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. respect to MAT scores and degree program (Master of Arts in 
Teaching, Master of Education (On-Campus) , Master of Education 
(Off-Campus) and Master of Science in Education) . A proportional 
sample of 162 students was selected across the 4 degree programs, 
and to assure enough students at the lower end of the MAT scale 
were present in the sample, a slight over-representation of 
students with MAT scores below 3 0 were chosen to be included in the 
study . 

An attempt was made to collect the written statement from the 
student's application for graduate study for each of the 162 
students. Application statements, which served as the writing 
sample for this study, could not be located for 20, or 12%, of the 
students. Thus, the final sample consisted of 142 students. Data 
for the remaining variables were supplied by the university's 
Office of Institutional Research. 

Data Sources 

Predictor Variables 

Junior/Senior Undergraduate GPA . This predictor was 

determined by averaging the grade point values (A =4, B=3, C= 
2, D = 1, F = 0) for all courses taken during the student's junior 
and senior years in undergraduate school . 

Miller Analogies Test (MAT) Score . The raw score from the 
Miller Analogies Test was used as a another predictor in this 
study. Raw scores on the MAT may range from 0 to 100. 

Writing Sample Score . As part of the admission process for 
graduate study, applicants complete a written statement, which in 
the past has been read, but has not been formally scored nor 
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, evaluated. This written statement has also received less weight 

than the junior/ s ^nior GPA and the MAT score in determining 

admission to graduate study. The following statement appears on 

the graduate application form: 

Please describe your reasons for wishing to complete 
the degree for which you are applying. Present some 
of your future professional plans and indicate what 
you hope to accomplish from this additional study. 

This statement is one of the criteria used to evaluate 
your application. 

The writing samples were scored independently by 3 readers who 
have had several years experience scoring essays which are used as 
an admission criterion for alternative teacher certification 
programs in Texas. The writing samples were scored according to 
the following 4-point scale which is a modification of the scale 
used to evaluate essays written by applicants to the Texas 
alternative teacher certification programs: 

4 - A well organized essay that demonstrates clear 

competence in writing, although it may have a few 
minor errors or occasional awkv/ardness of 
expression . 

3 - An essay that is well written but somewhat 

general and lacking in details. It may have 1 
or 2 major errors and some minor errors. 

2 - The essay may be too brief, underdeveloped or 
too general to deal adequately with the topic, 
or the essay may indicate problems in basic 
writing skills. 

1 - The essay displays serious faults in writing and 
weakness in content . 

To score the writing sample, each reader assigned one of the 
four score points to the essay and was also allowed to indicate a 
plus (+) or a minus {-) if the writing sample was at the top or at 
the bottom of a particular score category. In only 2 instances out 
of 142 did the scores of any two readers differ by more than 1 
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* point. In those two cases the differences were resolved so that 
all readers arrived at scores that were no more than 1 score point 
apart . 

To increase the variability in the essay score scale, the 
scores of the 3 readers were summed and multiplied by 4, and then 
one point v;as added to the result for each plus ( + ) given by the 
readers (maximum of 3 points) , and one point was subtracted from 
the result for each minus ( ) given by the readers (maximum of 3 
points) . Thus, the possible range of sco.es for the writing sample 
was 9 to 51. The interrater reliability for scoring the writing 
samples for this study was r = .92. 

Dependent Variables 

Graduate GPA . For each student in the sample the graduate GPA 
was determined by averaging the grade point values for all graduate 
courses taken in the College of Education, employing the same 
procedure used to determine the junior/senior undergraduate GPA. 

Graduation Success . If a student had graduated by June 1995, 
the student was assigned a code of 1 for the graduation success 
variable. If a student had not graduated by June 1995, the student 
was assigned a 0 for the graduation success variable. 

The final sam.ple consisted of 142 students (Female, 85%; Male, 
15%) from the following degree programs: Master of Education (Off- 

campus) , 53%; Master of Education (On campus), 25%; Master of Arts 
in Teachi.ng, 21%; and Master of Science in Education, 1%. The 
final percentage closely approximated the percentage of students 
actually admitted into each of the degree programs. Nearly 87% of 
the students in the sample had graduated within 4 years of taking 
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. their first course at the university. 

Means, standard deviations, and ranges viere determined for the 
MAT, the GPA variables and the writing score. (See Table l) . 



Table 1 

Means, Standard Deviations and Ranges 
for the Predictor and Dependent Variables 



Variable 



Mean SD 



Range 



MAT Score 




45 


19 


9 


to 


90 


Junior/Senior GPA 


3 , 


.01 


.47 


1.72 


to 


3.95 


Graduate GPA 


3 


. 71 


. 36 


2.60 


to 


4.00 


Writing Score 


30 


.84 


11 . 40 


11 


to 


50 



The mean MAT score reflected the ^slight over- sampling of 
students with lower MAT scores (sample mean = 45, population mean 
= 47 ) . A similar finding was also evident for the junior/senior 
GPA (sample mean = 3.01, population mean = 3.09) and graduate GPA 
(sample mean = 3.71, population mean = 3.86) . 

In addition, correlations among all the predictor and 
dependent variables were calculated. (See Table 2) . The only 
significant correlations (p < .01) between the predictors and the 
dependent variables were between graduate GPA and the following 
predictor variables: MAT score (r = .37), junior/senior GPA (r = 
.31) and the writing score (£ = .45) . 

A stepwise regression was performed using graduate GPA as the 
dependent variable and the MAT score, junior/senior GPA and the 
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• writing score as the predictors. The results of the regression 
analysis indicated that the writing score was the best single 
predictor of graduate GPA (R - .45) . After taking the writing 

score into account, the second best predictor was junior/senior 
GPA. With the addition of junior/senior GPA to the equation, the 
multiple R increased to .51. The MAT score did not add signifi- 
cantly to the prediction equation once the writing score and the 
junior/ senior GPA were taken into account. 



Table 2 

Correlations Between the Predictor and Criterion Variables 



Criterion Variables 



Program Graduate 

Predictors Completion GPA. 



Writing Score 


. 17 


.45** 


Junior/Senior GPA 


- .03 


.31** 


MAT Scores 


- . 03 


. 37** 



** p < .01; n = 142 

Educational Importance 

The data indicate that although none of the variables could 
accurately predict whether or not a student would complete the 
program within 4 years, the writing sample, in conjunction with the 
junior/senior undergraduate GPA, could be used with moderate 
success to predict graduate GPA for students in a masters program 
in a SchooZ, of Education. 

The results of this current study support the findings of the 
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. previous studies conducted at this university, namely that the 
writing score was a better predictor of graduate GPA than either 
the MAT score or undergraduate GPA. In addition, the correlation 
between the writing sample score and graduate GPA was fairly 
consistent across the three years that the study was conducted; 
r = .40 for 1993; r = .48 for 1994; and r = .45 for 1995. 

This study provides additional evidence that the use of 
authentic methods, such as a written statement, may be valuable in 
assessing the ability of students to experience success with 
masters work in Education. Additionally, the writing sample could 
provide valuable diagnostic information to help students improve 
their writing skills so that they may be more successful in their 
graduate studies . 
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